Laboratory tests: automatic systems

A laboratory is a sheltered place where one constrains the enthusiastic abandon of the real world in order to attempt to lay the blame for one apparently capricious phenomenon upon another. The extraordinary difficulties experienced in all areas of science when designing experiments testify to the complexity of the interactions between natural phenomena. Experiments are concerned with relationships between events or states as reflected by the measurements that we choose, and are able, to make. It is invariably necessary to control, or at least to keep tabs on, factors which are not the immediate concern of the experiment, but are suspected to have some influence on those that are. These comments apply whether an experiment is conducted in laboratory conditions or not. What, then, are the characteristics of a laboratory experiment which distinguish it from a real life experiment? I do not think that there is a clear distinction. However, one might point to a tendency for the experimenter to initiate events at his convenience in the laboratory, whereas he will observe natural occurrences in real life. In a laboratory experiment it will often be possible to control subsidiary variables: in a real life experiment one is more often faced with the problem of eliminating them in the analysis. It is easy to find exceptions to these tendencies. In experiments on aspects of human behaviour, for example, it is not always possible to control variables as the researcher would wish, even within the laboratory. Human cognitive activity is a substantial component of information retrieval, and in another chapter of this volume, Keen has discussed the problems of laboratory experimentation on that aspect of the field. In contrast, this chapter will consider laboratory work on the mechanical components of information retrieval. The events that the experimenter observes, in this case, are all executed by a machine whose characteristics can, to a large extent, be determined by the experimenter himself. Thus, in principle, he can exercise a considerable degree of control over variables. Indeed, it is hard to think of a 'purer' laboratory environment in science. The most substantial examples of the type of work to which I refer are the long series of tests with the Smart system (see Chapter 15) and those conducted over a period of several years at Cambridge University by Karen Sparck Jones, C. J. van Rijsbergen and others.